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Abstract 

Critical thinking extends to all aspects of professional engineering, especially in technical development, and, since 
the introduction of the ABET 2000 criteria, there has been an increased emphasis in engineering education on the 
development of critical thinking skills. What is hoped for is that the students obtain critical thinking skills to 
complement their abundant logical thinking skills. Critical thinking is a developmental skill that cannot be taught 
simply by traditional methods, i.e., step-by-step instruction followed by repetitive skills, as used for logical thinking 
skills. The research and development reported here is concerned with outlining how students first can be alerted to 
what critical thinking is, and how critical thinking can be nurtured through practical experience with appropriate 
guidance and reinforcement. The module ‘Introduction to Fluid Mechanics’ has been selected to demonstrate how 
formal critical thinking components may be introduced with the development of principles of logic and an obligatory 
problem solving procedure. A method of assessment during various phases of critical thinking, i.e., initial design or 
set-up, testing of method or prototype, and evaluation of the results, of open-ended assignments in engineering for 
each year of the undergraduate course has been developed. 

Keywords: Critical thinking. Undergraduate, Learning environment. Assessment 

1. Introduction 

The quality of students’ and engineers’ thinking, and how they think, determines the quality of what they design, 
produce or make. Critical thinking, it is claimed, in engineering education occurs today mostly in a focused context, 
directed toward fulfilling one of several ABET learning outcomes (Claris & Riley, 2013). Of course the teaching of 
critical thinking should be more than this, as critical thinking entails much more than the conventional practices in 
engineering, by considering and articulating assumptions in problem solving, selecting appropriate hypotheses and 
methods for experiments and structuring open-ended design problems. Just as is found in many professions, 
engineers now need, more than ever, strong critical thinking skills to deal with a world of increasingly rapid change 
and complexity. Much has been written in the literature in the contexts of problem solving (Lombard, 2004; Mina et 
al., 2003; Papadopoulos et al., 2006), conducting experiments (Bruno & Anderson, 2005; Miller & Olds, 1994) and 
open-ended design (Gurmen et al., 2003; Lunt & Helps, 2001). Attention has also be given to critical thinking within 
topics in engineering such as ethical decision making (Swailie & Kreppel, 2001; Wolverton & Wolverton, 2003), 
social impacts of technology (Nelson, 2001), and positioning students and engineers from a stance of social justice 
(Baillie, 2013). 

This work is concerned with examining what it means to teach critical thinking, and, how this can be achieved 
effectively. There is a definite need to help students develop higher-order thinking skills, i.e., to do something more 
sophisticated than recite back facts they have memorized from lectures or textbooks, sometimes with little required 
analysis (Bissell & Lemons, 2006). What is hoped for as the outcome to this work is a major change in how students 
respond to problems and to give them the tools to think critically. Justification for the inclusion of critical thinking in 
the curriculum is well supported in the literature and by many national commissions (American Association for the 
Advancement of Science, 1989; Felder, 1987; National Academy of Sciences, 2003; Yuretich, 2004). Yet when 
trying to implement critical thinking as an explicit goal in undergraduate engineering, there does not seem to be a 
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well-defined scheme to ensure success. This may be in part, due to a lack of a good method to assess improvement in 
critical thinking (Bissell & Lemons, 2006). 

It has been concluded that most students already consider themselves as effective critical thinkers, and faculty claim 
that critical thinking is a primary objective of their course, which may or may not be self-delusion (Paul et al., 1997). 
Also, it has been quickly recognized that although workshops and seminars on critical thinking have their place to 
initiate awareness, a systematic semi-implicit approach over a period of time is very necessary to cultivate critical 
thinking (Felder, 1984). That is, once the principles of critical thinking are grasped and, importantly accepted by the 
student, then a period of time is necessary for cultivation, practice and hopefully becoming second nature. 

To improve, or even to initiate critical thinking, a model first needs to be developed, capable of analyzing and 
evaluating the way engineers think, and hence this model applied appropriately. The general consensus found in the 
literature is that an explicit initial course on developing critical thinking skills is needed followed by reinforcement 
(Niewoehner, 2006; O’Shea et al ., 2012). It is this type of approach which is developed here, where the critical 
thinking course is followed by a preliminary initiative to cultivate more and systematic critical thinking within 
various engineering modules found in an undergraduate engineering course. There is also some attempt to assess any 
improvement in the students’ ability to think critically. 

In the rest of this paper, a discussion will be given of critical thinking in general, comments on developing an initial 
course on critical thinking, and how critical thinking may be reinforced throughout the rest of an undergraduate 
engineering course. An assessment method of student critical thinking development as they progress through their 
undergraduate engineering course is included. A section is also included summarizing the results of a discussion on 
how students view critical thinking in relation to themselves and how necessary they feel critical thinking is. Results 
of an experiment using a pre- and post-test control group design to analyse the effect of the inclusion of critical 
thinking components in the undergraduate module ‘Introduction to Fluid Mechanics’ are reported. 

2. Initial Thoughts on Critical Thinking 

To solve tough problems of society, especially when they are technological, creative engineers are needed (Felder, 
1987). It would seem that the responsibility of schools of engineering in universities, and also in society’s best 
interest would be to create creative engineers. Part of the problem of not dealing with critical thinking at 
undergraduate level is a tendency to push more and more information into each module so that just covering the 
syllabus material is all there is time for. Also faculty do not get personally rewarded for innovative teaching at the 
same level as research (Felder, 1984) so providing little incentive other than personal satisfaction. While there are 
many innovative ideas and trends within today’s teaching (Adair et al., 2014; Felder et al., 2000; Adair & Jaeger, 
2013; Seymour, 2002), there is still the tendency to over-use the lecture-homework-quiz format. This is indeed an 
efficient method of delivering knowledge and skills, but it has never been shown to be effective at producing the 
critical, innovative thinking skills needed to solve difficult technological problems. For critical thinking to be 
included in a meaningful way within engineering education undergraduate teaching and learning, it is argued that the 
following premises could give a basis for development: 

• Defining what ‘critical thinking’ actually means and entails. 

• Certain techniques have been identified by education theorists and psychologists, which have the possibility 
of stimulating creativity if integrated properly into the more traditional forms of instruction. 

• While it is recognized that an initial course may be desirable/needed on critical thinking, the techniques 
mentioned above must be introduced throughout the curriculum. 

• The methods intended to develop creative/high-level thinking must not take up too much time. As much 
integration as possible, with the existing syllabus, is desirable. 

• Establishing a method to assess, however crudely, to measure ‘critical thinking ability’. 

• Faculty should form workshops to discuss the sharing/development of techniques designed to encourage 
critical thinking. 

An integral part of teaching and learning is the ability to be able to assess its effectiveness. When it comes to 
assessing critical thinking, many faculty in fact have expressed the view that critical thinking cannot be achieved or, 
there is no current satisfactory method for assessment (Beyer, 1984; Cromwell, 1992; Aviles, 1999). However, there 
is also evidence from a large body of literature showing critical thinking can indeed be assessed, although careful 
selection of the assessment method may be warranted (Cromwell, 1992; Fisher, 1997) although most of this 
assessing is subject-independent, i.e. the measurement of critical thinking ability has no regard for the context. Such 
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testing may have problems, including: the measurements may not indicate anything useful about discipline-specific 
knowledge; administering such tests takes time away from the course (can be viewed as wasted time); faculty lack 
time to really get to understand the structure and theory behind the tests (Bissell & Lemons, 2006). Rather, it is 
thought preferable to develop assessment methodology which includes both content and cognitive skills testing, and, 
in which the critical-thinking component can be explicitly identified and then scored. 

3. Designing a Critical Thinking Model 

Building on the premises listed in the last section, a summary of the research is now given which contributed to a 
course on critical thinking plus a strategy to implement critical thinking throughout an engineering undergraduate 
course. 

3.1 What is Critical Thinking 

Different definitions of the term ‘critical thinking’ abound in the literature (Brookfield, 2012; Facione, 2015; O’Shea 
et al., 2012; Aretz et al., 1997) and this variability may in fact impede progress on all fronts (Resnick, 1987). Most of 
the definitions share some basic features and probably address some aspect of critical thinking (Potts, 1994). In the face 
of so many definitions, some authors go for a ‘consensus definition’ and others go for one that meets their needs and 
consistency in applying it (Bissell & Lemons, 2006). For example, Baillie (2012) explains critical thinking from the 
point of view of outcomes and at the beginning of her course she explains “students will be able to demonstrate an 
ability to think critically and reflexively not only about engineering practices in the abstract but about their own work 
in this unit; assess and apply different views of the relationship between science, technology and society; consider 
rights, justice, freedom and ethics and illustrate their relation to engineering practice; and compare and critique local 
and global technological practices”. It can be taken from this lengthy definition that critical thinking goes beyond 
thinking clearly or logically and actually looks in depth at various relevant concepts by exploring such things as 
underlying issues, loosely connected issues or seemingly unconnected issues. 

However, important to critical thinking is how to assess it. With this in mind, a definition, and the working one for this 
paper, based on Bloom’s taxonomy of educational objectives (Bloom, 1956), was chosen. Bloom’s taxonomy 
delineates six categories of learning basic knowledge, secondary comprehension, application, analysis, synthesis and 
evaluation, where the first two categories do not require critical thinking, but the last four all require higher-order 
thinking which characterizes critical thought. Breaking down academic skills in this way and to some extent 
identifying critical thinking provides a reasonable breakdown for designing specific assessment designs. The 
assessments developed however cannot be used to measure critical thinking independent of content (Bissell & Lemons, 
2006). 

3.2 Initial Critical Thinking Course 

An initial course to introduce students to ideas which should improve their thought was devised. The course is based on 
Paul’s critical thinking model (Paul & Elder, 2002). The model, shown on Figure 1, has the goal of producing a mature 
engineering thinker. 



Figure 1. Paul’s critical thinking model (Paul & Elder, 2002) 
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The critical thinking course lasted one semester in the engineering undergraduate degree preparatory year (Level 0) 
and consisted of lectures, group discussions, and practicing analyses. Engineering design was used as the vehicle to 
introduce elements of critical thinking. 

The course actually starts with the third grouping, i.e. lecturing on discussing ‘Intellectual Traits/Values’ in relation to 
engineering practice, followed by an introduction to ‘The Elements of Thought’, where students start analysing how 
they think and how others think. Lastly the ‘Essential Intellectual Standards’ are lectured on, discussed and practiced 
using exercises based on engineering design. Each of these sections is now briefly discussed. 

3.2.1 Intellectual Traits/Values 

The overall objective here is to make students aware that professional engineers must cultivate personal and 
intellectual values, so that they think with insight and integrity (Paul & Elder, 2002). For engineers certain distinct 
issues come to the fore within each category: Intellectual humility admits to ignorance and being truthfully sensitive to 
what you know and what you do not know. Intellectual autonomy is the ability to think for oneself while adhering to 
standards of rationality which is important so as not to accept other viewpoints without questioning. Intellectual 
integrity means that you do not have double standards and you hold yourself to high intellectual standards expected by 
you in others. Intellectual courage is when you have strong views, which may be unpopular, and you are willing to 
express these views to your peers. Intellectual perseverance is the ability to work through complex and frustrating 
tasks. Confidence in reason is a mixture of being open-minded regarding other peoples’ views, encouragement of other 
people to have a view and the ability not to distort views to support my own position. Intellectual empathy is important 
as it helps the engineer grow and mature as he/she takes on different views. Fair-mindedness is the ability to treat all 
viewpoints without bias or prejudice and so be able to make judgments appropriately. The method of delivering this 
part of the course was to first give an introductory lecture followed by small group discussions using appropriate lists 
of questions. At the end of each session, the complete cohort came together to summarise the findings. 

3.2.2 The Elements of Thought 

In this section of the course the aim was to help students evaluate how they think, or how others think (Paul & Elder, 
2002). The reasoning behind the structure of this part of the course is that whenever we think, we think for a purpose 
within a point of view, based on assumptions, leading to implications or consequences. We use data, facts and 
experiences, to make inferences and judgments, based on concepts and theories, to answer a question or solve a 
problem. A similar approach for delivery was used as for the first section of the course, where introductory lectures 
were given, followed by discussions and exercises based on engineering design. 

3.2.3 Essential Intellectual Standards 

Such standards need to be applied when one is interested in checking the quality of reasoning about an engineering 
problem or project (Paul & Elder, 2002). To think professionally as an engineer means having command of all of these 
standards. As with the last section delivery here was by introductory lectures, followed by discussion and exercises 
based on engineering design. A brief list of typical questions associated with each of the intellectual standards is given 
in Table 1. 
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Table 1. Typical design questions associated with intellectual standards (Paul & Elder, 2002) 


Intellectual Standards 

Design Questions 

Clarity 

- What are the success criteria? 

- Have assumptions been clearly defined? 

Accuracy 

- Are the modelling assumptions appropriate? 

- Has the test equipment been calibrated? 

Relevance 

- Does the design address the requirements? 

- Has irrelevant data been included? 

Logical validity 

- Are design decisions based on appropriate analysis? 

- Are there any hidden assumptions? 

Breath 

- Have alternative approaches been considered? 

- Have end-of-life issues been considered? 

Precision 

- What are the accepted tolerances? 

- What are the error bars or confidence bounds? 

Significance 

- What are the design drivers? 

- What impact will there be on the market? 

Completeness 

- Is there room for further development? 

- How could the next version be improved? 

Fairness 

- Are vested interests influencing the design? 

- Has public/community interests been considered? 

Depth 

- How far have the complexities been accounted for? 

- Has growth capability been addressed? 


3.3 Reinforcement Throughout the Undergraduate Course 

In addition to the ‘Initial Critical Thinking Course’, it is thought imperative to integrate critical thinking into each of 
the modules found in the engineering undergraduate course. The support of faculty members is indispensable here, as 
they have the power to include critical thinking or not in their modules. Seminars/discussions were held for the faculty 
on the inclusion of critical thinking in the classroom at the beginning of the academic year and several times during a 
semester derived from the theory put forward by Glaser (1941), who has suggested five components critical to the 
reinforcement of critical thinking. 

• The first component, specific knowledge, was based on the belief that knowledge was necessary for critical 
thinking. Engineers must have a knowledge base on which to build their critical thinking skills. 

• The second component is experience. Engineers have the know-how that allows for instantaneous 
recognition of patterns and intuitive responses in expert judgment. 

• The third component is competencies. General critical thinking competences are related to the scientific 
process, hypothesis generation, problem solving, and decision-making. 

• The fourth component is attitude, including confidence, independence, fairness, responsibility, risk taking, 
discipline, perseverance, creativity, curiosity, integrity, and humility; and intellectual standards, including 
clarity, precision, specificity, accuracy, relevance, plausibility, consistency, logicality, depth, broadness, 
competence, significance, adequacy, and fairness. 

• Finally the fifth component is professional standards so as to accentuate safe, competent engineering 
practice. 

Three levels of critical thinking were identified in the seminar. An initial level where answers to complex problems 
are right or wrong and there is only one right answer. A second level was the complex level where students could 
recognize options and alternatives, but did not make a commitment to any one solution. The final level, and ultimate 
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goal, was commitment by the students to a solution. At this level, students (engineers) must be encouraged to choose 
an action or belief based on the options identified at the complex level. 

The faculty were then encouraged to think about the following so as to facilitate better integration of critical thinking 
skills into their teaching and learning as outlined in Table 2 (Purves, 2009): 

Table 2. Factors that influenced the development of critical thinking skills 

Pedagogical Factors that Influence Critical Thinking Development 

Curriculum Design - Acquiring foundational concepts 

- Progressing from simple to complex concepts 

- Applying learning in the laboratories/projects 

Integrative Learning Activities - Tests 

- Case studies 

- Simulations 

Personal Factors that Influence Critical Thinking Development 

Curiosity 

Confidence 

Perseverance 

Other Factors that Influence Critical Thinking Development 
Faculty support 

Reinforcement, both in and out of the engineering programme 
3.4 Assessment of Critical Thinking 

Current best practices in critical thinking for engineering education have already been surveyed (Cooney, Alfrey & 
Owens, 2008), where two major themes emerged: Open-ended problem-solving assignments with no clear-cut “right” 
answer or approach, and written assignments with a reflective component, frequently requiring judgment in the face of 
uncertainty. Problem solving (Cloete, 2001) and the components of a complete act of thought as proposed by Dewey 
(1910) have been compared by Cooney et al. (2008). Dewey’s work forms the foundation for current literature on 
critical thinking. 

It is not the case that good assessment should be just the addition of assignments that draw on critical skills only. Such 
subject-independent assessments of critical thinking have indeed been developed and used (Cromwell, 1992; Fisher & 
Scriven, 1997), but acceptance that such testing is useful is questionable in that faculty can doubt that the 
measurements indicate anything useful about discipline-specific knowledge, administering such tests takes time away 
from the content of the course and can be costly, and, most faculty lack the time to learn the underlying structure and 
theory behind the tests (Bissel & Lemons, 2006). With such practical problems, the assessment methodology 
developed here allows for discipline-specific, i.e., content-based questions within which the critical thinking 
component can be explicitly dissected and scored. The work is built on the work of others who have used Bloom’s 
taxonomy to drive assessment decisions by using this taxonomy to explicitly define the skills that are required for each 
question. 

3.4.1 Designing Discipline-Specific Assessments of Critical Thinking 

The methodology consists of several steps in that questions which require both engineering knowledge and critical 
thinking skills are written followed by the documentation of the particular content and critical thinking skills required. 
A scoring rubric is then devised for the question. The scheme used here is built on the work of others (Porter, 2002, 
Middendorf & Pace, 2004, Facione et al., 2000). The questions are reviewed by other faculty and the assessments are 
administered to the students and scored against a rubric that has been established in advance. It was found that rather 
than changing existing questions it was much more efficient to write new questions. 

It was important to alert the students that expectations for the answers to these questions are different to what they may 
meet normally. The methodology has varying degrees of open-endedness in the answers while the questions are akin to 
more advanced questions found in engineering in that the students must analyse several concepts simultaneously to 
reach a reasonable answer. The students must be aware that a substantial part of the marks awarded are given to the 
rationale presented in the answers. The first step in evaluating the effectiveness of the problems is to clearly define the 
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expectations for each question using the components of the framework of Ebert-May et al. (2003). These expectations 
are valid when the student gets the correct and complete answer, or when the student answers the question by drawing 
on the expected content. However, it is possible to apply the correct critical thinking skills to these problems while 
getting some aspects of the content wrong or employing an alternative body of content (Bissell & Lemons, 2006). This 
is key to an assessment technique and a grading rubric (Bissell & Lemons, 2006) was designed. 

As with any short-answer examination, it is much easier to grade when students present clear answers combined with 
good reasoning. However, many students present answers which take a long time to decipher. Here, as the range of 
acceptable answers have been predefined through expectations, this problem is reduced. 

4. Including Critical Thinking Components in a Module 

There are many approaches to including critical thinking components in an engineering module (Mourtos et al., 2004), 
with the approach of Woods et al., 1997, chosen here as suitable for solving problems within the module ‘Introduction 
to Fluid Mechanics’. Woods et al., 1997, state that for critical thinking and hence enhanced problem solving 
capabilities, students must have the attributes listed in Table 3. Here the affective levels are, receiving (a stimulus), 
responding (to a stimulus), valuing (an object or a behaviour), organisation (of values into a system) and 
characterisation (by a value complex). The cognitive levels are knowledge (recognise/recall information), 
comprehension (understand the meaning of information), application (use information appropriately to solve 
well-defined problems), analysis (deal with ambiguity with new ill-defined situations, formulate models), synthesis 
(combine elements in novel ways to generate new products or ideas) and evaluation (judge the worth of ideas, theories 


and opinions). 

Table 3. Attributes needed for enhanced problem solving 


Attribute 

*Bloom’s taxonomy 

1 . 

Time is spent reading, gathering information and defining problems 

Affective - level 2 

2. 

Develop a process as well as having a few tactics and heuristics 

Cognitive - level 4 

3. 

Monitor any problem-solving process and reflect on its effectiveness 

Cognitive - level 4 

4. 

Emphasis accuracy as opposed to speed 

Affective - level 3 

5. 

Write down ideas and use charts/figures 

Cognitive - level 3 

6. 

Be organised and systematic 

Affective - level 4 

7. 

Be flexible and view the situation from different perspectives 

Affective - level 4 

8. 

Draw on pertinent subject knowledge and critically assess the quality, accuracy and 
pertinence of that knowledge/data/results 

Cognitive - level 3 

9. 

Be willing to risk and cope with ambiguity, welcoming change and managing stress 

Affective - level 4 

10. 

Use an overall approach that emphasises fundamentals, rather than trying to combine 
various memorised sample solutions 

Cognitive - level 4 


*(Bloom, 1984; Bloom et al., 1984)) 


An approach traditionally used in many engineering curricula, i.e. teaching and learning heavily relying on exercise 
solving, where students develop solutions mainly based on solutions they have seen before, was taught to all students 
for approximately three quarters of the course. For the last quarter of the course the cohort was divided into two, with a 
control group continuing using the traditional approach, and, with an experimental group looking at problems with 
more critical thinking involved. This initial approach to including critical thinking in the module was to assess if any 
improvement was found in student results as discussed in subsection 5.1 below. In future it is intended to include 
critical thinking from the beginning of the module. The difference in the two types of course delivery dining the last 
quarter of the module is summarized in Table 4. 
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Table 4. Including critical thinking components verses the traditional exercise approach 

Critical thinking delivery Exercise solving delivery 

1. Emphasises best answer to an unknown Emphasises only one right answer 

2. Include ambiguity within given information Well defined situation with all information explicitly given 

3. New situations given Building on similar situations previously encountered 

4. No clues as to what knowledge or skill needed Hints and prescribed assumptions often given 

5. There could be more than one valid approach One approach gives correct answer 

6. The method of solving is not clear Build on familiar solutions 

7. Knowledge necessary to solve problem can Problems tend to come from one subject only, and even only 

come from several subjects one topic 

g Required good communication skills Communication skills not often required as equations/graphs 

are all that is necessary 

To give a framework to help the students be systematic in their approach to critical thinking, the method suggested by 
Woods, et al. , 1997 was adopted. This is summarised in Table 5. 

Table 5. Critical thinking methodology 

Step Description 

0. Motivation I can do it 

I want to do it 

1. Definition of problem Define what the problem states 

Sketch the problem 
Determine appropriate information 
Determine any constraints 
Define criteria forjudging answer(s) 

2. Explore the problem What is the real objective? 

What are the issues? 

What would be reasonable assumptions? 

Give an approximate answer 

3. Plan the solution Develop overall plan 

Develop any sub-problems 

Select appropriate theory, principles and approach 
Determine any research that needs to be done 

4. Implement the plan 

5. Check the solution Check accuracy of calculations 

Check units of calculated parameters 

6. Evaluate/reflect Is the answer reasonable? Does it make sense? 

Were the assumptions good? 

How does the solution compare with the approximate answer 
of (2) above? 

When appropriate - is the solution ethically sound? 

To assess the students’ answers to questions, which included critical thinking, in the ‘Introduction to Fluid Mechanics’ 
module, a scoring rubric as shown in Table 6 was developed. What was looked for was a listing of given information, 
succinctly paraphrasing the problem statement, identifying assumptions, drawing a schematic, identifying the general 


Published by Sciedu Press 


30 


ISSN 1927-6044 E-ISSN1927-6052 




www.sciedupress.com/ijhe International Journal of Higher Education Vol. 5, No. 2; 2016 


governing equation(s), analysing the problem by simplification of the governing equations and performing necessary 
calculations, highlighting the results and drawing conclusions. 

Table 6. Scoring rubric for questions involving critical thinking 


3 

2 

1 

0 

Givens 

All givens stated and 

All givens stated but 

Given list not 

Givens not explicitly 


appropriately 
selected; units given 

not appropriately 
selected; units given 

complete 

mentioned 

Problem statement 

Problem statement is 

Problem statement is 

Problem statement is 

Problem statement 


complete and clear 

clear but missing 
important facts 

ambiguous 

missing 

Assumptions 

Assumption list is 

Some assumptions 

Most assumptions 

No assumptions 


complete 

are missing or 
confusing 

Drawing exists but 

are missing 

given 

Schematic 

Drawing is clear and 
well labelled 

not clear 

Some key equations 

Ambiguities exist 

No relevant drawing 
given 

Governing equation 

All relevant 
governing equations 
given 

missing or wrong 

Some errors exist 

Most key equations 
are missing 

All given equations 
are wrong 

Analysis 

Equations are 
reduced according to 
assumptions, and 
algebraically 
manipulated. Steps 
are easy to follow. 
Units dealt with 
correctly 

Results given but not 

Many errors exist 

Analysis is irrelevant 
or unreadable 

Results 

Results are clearly 
indicated, correct 
and thorough 

clearly highlighted 
or some results 
missing 

Correctness of the 

Results are 
incomplete or 
difficult to identify 

No results given 

Conclusions 

Correctness of the 

results is judged. 
Validity of answer in 
terms of assumptions 
is discussed 

Wrong results are 

No conclusions are 


results is judged. 
Validity of answer in 
terms of assumptions 
is discussed 

improperly judged to 
be reasonable, or 
assumptions not 
reflected on 

drawn 


5. Feedback on the Critical Thinking Model 


The following section is divided into four sub-sections. The first subsection reports on an experiment using a pre- and 
post-test control group design (Pfahl et al., 2004). The next three each report feedback solicited using an online 
questionnaire, or, by group discussions with selected students and faculty concerning the critical thinking model 
implemented. The feedback is based on the first cohort using this critical thinking model. 

5.1 Evaluation of Including Critical Thinking Components in a Module 

An evaluation of including critical thinking components in the undergraduate module, ‘Introduction to Fluid 
Mechanics’ based on a controlled experiment (Pfahl et al., 2004) was carried out after the authors’ ethics committee 
approved the study. The experiment used a pre- and post-test control group design. The students had to undertake two 
tests, one after about three-quarters of the teaching process had been given (pre-test) and one after the teaching process 
was fully complete (post-test). The experimental group were exposed to the critical thinking approach to solving 
problems during the last part of the course, whereas the control group was not. Learning effectiveness was evaluated by 
comparing within each group post- to pre-test scores and by comparing the scores between students of the experiment 
group and students of the experimental group and students of the control group to find relative and absolute learning 
effectiveness. 
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In order to measure students’ performance the students had to answer five open-ended questions with each question 
requiring careful critical analysis. The open-ended nature of questions was chosen in order to avoid mere guessing 
which usually takes place to an unidentifiable extent with multiple-choice tests. The teaching for the two groups 
(experiment - with critical thinking components - group A, control - without critical thinking components - group B) 
was delivered using traditional lectures, experiments and tutorials. The five test questions (scenarios) used in the 
experiment are listed in the Appendix. 

The experimental hypotheses for the dependent variables were as follows: 

1. Both groups (A: experimental group, B: control group) experience a positive learning effect and post-test scores 
are significantly higher than pre-test scores. 

2. Group A experiences a higher relative learning effect than group B with regard to the improvement between pre- 
and post-test, and a higher absolute learning effect with regard to the post-test performance. 

5.1.1 Method 

For the experimental group A (n = 26) and group B (n = 25) of undergraduate students in their third semester were 
chosen. This was the first time in their engineering program they had met fluid mechanics as a topic and the groups 
were formed using the combined results of an earlier thermodynamics course so meeting the requirements for a 
quasi-experiment (Beins, 2009). Students who underwent the pre-test did not know that they would undergo a post-test 
after the teaching process and they were not allowed to keep the questions nor did they receive correct answers after the 
tests. The pre- and post-test procedures were introduced by explaining that the tests were not part of the formal 
assessment and that results would not influence their final grade. 

The experimental hypotheses for the dependent variables were stated as follows: 

1. Both groups (A: experimental group, B: control group) experience a positive learning effect and post-test scores 
are significantly higher than pre-test scores. 

2. Group A experiences a higher relative learning effect than group B with regard to the improvement between 
pre-test and post-test, and a higher absolute learning effect with regard to the post-test performance. 

In order to investigate the effect of the teaching effectiveness on the dependent variables Y.l to Y.5 (see Appendix), 
standard significance testing was used. The null hypotheses were articulated as: 

• H 01 - there is no difference between pre- and post-test scores with group A and group B. 

• H 0 2a - there is no difference in relative learning effectiveness between group A and group B. 

• H 0 2 b~ there is no difference in absolute learning effectiveness between group A and group B. 

Following Pfahl et al. (2004), hypothesis H 01 was tested by applying a one-way paired t-test, whereas a one-sided 
t-test was the appropriate test for the independent samples of H 0 2a and H 0 2b . A test to check the assumption of a 
normal distribution of the variables was conducted since the existence of a normal distribution is a prerequisite for 
applying the t-test. Although no normal distribution of the variables of the test samples could be assumed, most scores 
were distributed within the range of ±1 standard deviations around the samples’ means. No outliers were detected. 
Following the usual practice, the significance level a was set at 0.05. 

5.1.2 Results 

In the following the descriptive statistics and the results of the statistical analysis of the experimental hypotheses are 
presented. 

Table 7 summarises the descriptive statistics with pre- and post-test scores of the dependent variables for groups A and 
B. The last two sections of Table 1 show the calculated mean, median and standard deviation of the differences 
between post and pre-test scores. 
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Table 7. Scores of dependent variables 




Y.l 

Y.2 

Y.3 

Y.4 

Y.5 

Group A 

Mean 

1.520 

1.438 

1.125 

0.897 

0.617 

pre-test 

Median 

1.227 

1.354 

1.048 

0.543 

0.211 

scores 

Stdev 

0.912 

1.124 

0.988 

0.545 

0.333 

Group A 

Mean 

3.121 

3.012 

2.775 

1.436 

0.968 

post-test 

Median 

3.329 

2.930 

2.623 

1.500 

1.013 

scores 

Stdev 

1.567 

1.769 

1.877 

0.986 

0.955 

Group B 

Mean 

1.499 

1.446 

0.897 

0.751 

0.555 

pre-test 

Median 

1.312 

1.200 

0.888 

0.568 

0.243 

scores 

Stdev 

1.081 

1.007 

0.769 

0.521 

0.419 

Group B 

Mean 

2.211 

2.893 

1.989 

1.246 

0.797 

post-test 

Median 

2.222 

2.700 

2.001 

1.622 

1.083 

scores 

Stdev 

1.832 

1.921 

1.512 

1.072 

0.745 

Group A 

Mean 

1.601 

1.574 

1.650 

1.346 

0.351 

Difference 

Median 

2.102 

1.576 

1.575 

0.957 

0.802 

scores 

Stdev 

1.502 

1.034 

1.025 

1.098 

0.622 

Group B 

Mean 

0.712 

1.447 

1.092 

0.495 

0.242 

Difference 

Median 

0.910 

1.500 

1.113 

1.054 

0.840 

scores 

Stdev 

0.704 

1.399 

0.975 

0.784 

0.437 


Table 8 shows the results of the H 01 testing for each dependent variable (separately) based on a one-tailed t-test. 
Column shows the Cohen’s d effect size, column three the degrees of freedom (DOF), column four the t-values, 
column five the critical values when a = 0.05(Crit. t ogo ) and column six the associated p-value. Columns four and 
six show that group A achieved statistically significant for all variables. Also, the first two variables indicate high 
practical significance whereas Y.3 is in the boundary region of moderate/ high practical significance and Y.4 and Y.5 
show moderate practical significance. All variables indicate positive learning effectiveness. 


Table 8. Results for post-test versus pre-test for group A 


Variable 

d 

DOF 

t-value 

Crit. t 0 . 90 

p-value 

Y.l 

1.249 

25 

3.123 

1.316 

0.0022 

Y.2 

1.062 

25 

2.655 

1.316 

0.0068 

Y.3 

1.100 

25 

2.750 

1.316 

0.0055 

Y.4 

0.677 

25 

1.692 

1.316 

0.0515 

Y.5 

0.491 

25 

1.227 

1.316 

0.1156 

Using the same structure as in Table 8, Table 9 displays the results for control group B. The case for positive learning 
effectiveness is less obvious when compared to the results of Table 8. The variables Y.2 and Y.3 have p-values less 
than 0.05, which is a standard indication of statistically significant results for these two variables and they also have 
Cohen’s d values indicating moderate to high practical significance. The results for variables Y.l, Y.4 and Y.5 
indicate low to moderate practical significance. 

Table 9. Results for post-test versus 

pre-test for group B 




Variable 

d 

DOF 

t-value 

Crit. t 0 , 90 

p-value 

Y.l 

0.473 

24 

1.134 

1.318 

0.1340 

Y.2 

0.943 

24 

2.261 

1.318 

0.0166 

Y.3 

0.910 

24 

2.182 

1.318 

0.0196 

Y.4 

0.587 

24 

1.408 

1.318 

0.0860 

Y.5 

0.400 

24 

0.959 

1.318 

0.1736 


The results when testing H 0 2a are shown in Table 10. Results for the variables Y. 1 and Y.4 show both statistically and 
practical significance while the result for Y.3 shows moderate practical significance. The variable Y.5 has a result 
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showing a small practical significance and Y.2 suggests low practical significance. Overall it can be argued however, 
that these results show difference in relative learning effectiveness between group A and group B and so H 02a can be 
rejected. 

Table 10. Results for performance improvement, group A versus group B 


Variable 

d 

DOF 

t-value 

Crit. t 090 

p-value 

Y.l 

0.757 

50 

2.650 

1.299 

0.0070 

Y.2 

0.103 

50 

0.361 

1.299 

0.3606 

Y.3 

0.558 

50 

1.953 

1.299 

0.0313 

Y.4 

0.892 

50 

3.122 

1.299 

0.0023 

Y.5 

0.202 

50 

0.707 

1.299 

0.2432 


Table 11 shows the results of the testing of H 0 2b which examines the absolute learning of both groups. The variables 
Y.l and Y.3 indicate moderate to large practical significance whereas Y.3 and Y.5 indicate small to moderate practical 
significance. The dependent variable Y.2 supports the direction of the expected positive learning effectiveness but with 
a very low practical significance. Generally it can be said that the experimental hypothesis holds regarding absolute 
learning effectiveness and H 02b can be rejected. 


Table 11. Results for post-test improvement, group A versus group B 


V ariable 

d 

DOF 

t-value 

Crit. t 090 

p-value 

Y.l 

0.534 

50 

1.868 

1.299 

0.0370 

Y.2 

0.064 

50 

0.225 

1.299 

0.4119 

Y.3 

0.461 

50 

1.614 

1.299 

0.0598 

Y.4 

0.185 

50 

0.646 

1.299 

0.2622 

Y.5 

0.200 

50 

0.700 

1.299 

0.2453 


5.1.3 Discussion 


Based on the results presented in the previous sub-section and additional observations, the outcome of the experiment 
is now interpreted. As reported the change from pre- to post-test scores for the experimental group A confirmed a 
statistically significant positive impact on learning effectiveness and to some extent practical significance on learning 
effectiveness. The results did show a trend of giving less significant effectiveness as the questions increasingly 
required more critical thinking to get an appropriate answer. The improvement in results for group B was not as 
impressive when considering positive learning effectiveness. However there was an improvement in learning between 
pre- and post-tests. 

Testing the performance of relative learning between groups A and B showed that all variables showed a statistical 
significant improvement with the advent of critical thinking components. The results are mixed though, in that for 
variable Y.2 and Y.5 there was only a small impact of relative learning effectiveness. A similar result was found for the 
absolute learning effectiveness. 

Some potential threats to the validity of the above results have been identified. Concerning internal validity a selection 
effect was avoided to some extent by equalising the ability of the groups using the results of a closely related module 
on thermodynamics. The experimental design (i.e. introducing the experiment to students directly before the 
experiment) avoided students dropping out. The maturation effect caused by familiarisation and maturing was avoided 
by not informing students ahead of time that there would be a post-test. However, the internal validity might have been 
limited by the different interaction density between groups A and B (instrumentation effect). Concerning the external 
validity it can be expected that the results of the study are to some degree representative for the category of students 
found here. Students with different educational backgrounds, fields of study and socio-economic contexts may lead to 
different results. The exploratory character of the research presented here needs to be recognised and further 
investigations might lead to more generalised results. 

5.2 Questionnaire Results of Initial Critical Thinking Course 

An online questionnaire as summarized in Table 13 was used as the basis for evaluating the initial critical thinking 
course and consisted of questions to which a five-point rating scale was attached. The ratings have an ordinal rank (i.e. 
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from low to high) and Table 12 shows the weightings from the student responses. From a total cohort of 171 students 
some 90-94 (n) responded. 

Table 12. Weighting factors in percentages used for ordinal rating scales 


Rating 

Ordinal Weighting (%) 

A 

100 

B 

75 

C 

50 

D 

25 

E 

0 


The results are summarized in Table 13. It can be seen that students tended to find the concepts of the course fairly easy. 
This was a very important finding in that the students must understand what critical thinking is in order to apply critical 
thinking to challenging problems without worrying much as to how to apply it. It is obvious that assignments could be 
improved, probably by more grading of problems so that the learning curve employing critical thinking is lessened. 

Table 13. Number of student responses, n, and weighted scores for the questions 


Characteristics of 

course 

A 

weighting 

extreme 

E 

weighting extreme 

n 

Score 

(Mean) 

(%) 

Standard 

Deviation 

1. Rate the difficulty 
of the concepts 
covered in the course 

Very easy 

Very difficult 

93 

76 

5.6 

2. Rate the workload 
to complete this 

course 

Very little 

Very heavy 

90 

63 

4.8 

3. To what extent did 
the assignments 
contribute to a better 
understanding of the 
concepts? 

Helped a lot 

Totally useless 

93 

57 

7.2 

4. How well did the 

assessments reflect the 

course material? 

Very closely 

Not at all 

94 

64 

6.4 

5. What was your 
overall appraisal of 
this course? 

Excellent 

Poor 

94 

72 

4.6 

6. What was your 
attendance rate? 

All classes 

No classes 

93 

86 

3.5 


5.3 Discussion with Selected Students 

Twenty students were selected at random to give preliminary feedback on the impact of the above work so far in the 
form of a group discussion. As already alluded to, most of these students already thought of themselves as critical 
thinkers before the year started. This could be a widespread self-delusion, and it is clear work needs to be done to find 
out how true this self-opinion is. If it is untrue, then a process of changing attitudes and intellectual disposition, and, 
engendering a willingness on the part of the students to dedicate themselves more to thinking critically must ensue. 

The majority of students thought the initial course on critical thinking quite cumbersome and unnecessarily long. They 
thought that a more condensed message, more dedicated to explaining and reinforcing the ‘Essential Intellectual 
Standards’, may be more effective. They explained that at the end of the day they need a brief list of rules and processes 
to go by, to give a manageable framework to work within. 
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Certainly the students recognised that critical thinking is a process and with practice and the correct attitude they would 
become much better at it. They all stated that more critical thinking within each engineering module they took must be 
increased. 

5.4 Discussion with Selected Faculty 

Interviews were also conducted with twelve faculty, i.e. three from each of the engineering disciplines, mechanical, 
chemical, electrical & electronics and civil. The faculty were randomly chosen from a total of 62 faculty and the survey 
was in the form of individual discussions. The following are a summary of these discussions. 

The chosen faculty as a group were quite enthusiastic about the inclusion of critical thinking as an integral part of their 
respective courses but were not enthusiastic about testing critical thinking as a separate entity. There was general 
agreement that critical thinking should not be separated from the course material, as otherwise valuable time would be 
taken away from teaching and learning course content. Some of the staff expressed the view that a series of short 
courses must be implemented to increase their skills in including critical thinking both in the delivery of the course and 
in how best to assess it. They thought there was not enough uniformity between staff, especially in the method of 
assessment. They also generally thought that for assignments and projects it was relativity easy to include critical 
thinking and most felt comfortable in doing so, but short-questions were difficult to assemble. 

6. Conclusions 

Methods to cultivate critical thinking within engineering education have been examined and to some extent 
implemented. Essentially three major innovations have taken place, the first being the establishment of an initial course 
to students on what critical thinking is, why it is important an engineer has such a skill and how to incorporate this skill 
into everyday engineering practice. The second innovation was the creation of a seminar/discussion group for faculty 
on integrating critical thinking into their respective engineering modules. The third innovation is the development of a 
method to assess critical thinking during several phases of open-ended assignments in engineering. The method is 
designed to track the development of critical thinking as a student progresses through the undergraduate curriculum. 
Student feedback so far has asked for a more streamlined initial course and has encouraged course material in each 
module to include opportunities for more critical thinking. 

The experiment presented here evaluated the learning effectiveness of including critical learning components in the 
module “Introduction to Fluid Mechanics’. The results showed a significant improvement in learning effectiveness 
when including the critical learning components. In future introduction of the approach emphasising critical thinking 
and problem-solving will be introduced from the beginning of the module. 
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Appendix 

The following questions were used to assess the critical thinking skills acquired by the two groups A and B during 
the teaching phase of the ‘Introduction to Fluid Mechanics’ module. There were five questions selected from White, 
1998 and Mourtos et al., 2004. The questions are graded so that critical thinking is increasingly used to get an 
appropriate answer, as the questions move from relatively familiar topics to topics not considered directly during the 
course and where the statement may deliberately appear ambiguous to students. 

Y.l The Stokes-Oseen formula for drag force F on a sphere of diameter D in a fluid stream of low velocity V, 
density p and viscosity p is 

F = 3npDV + 9npV 2 D 2 /16. 

For small particles at low velocities, the first term in the above equation is dominant; hence, F ~ KV where K is a 
constant. Suppose a particle of mass m is constrained to move horizontally from the initial position x — 0 with 
initial velocity V 0 . Show 

(a) that its velocity will decrease exponentially with time. 

(b) that it will stop after travelling a distance x = mV 0 /K. 

Y.2 Water enters the bottom of the cone on the figure below at a uniformly increasing average velocity V — Kt. If 
d. is very small, derive an analytic formula for the water surface rise h(t) for the condition h = 0 at t = 0. 
Assume incompressible flow. 



Y.3 The device on the figure below is called a rotating disk viscometer. Suppose that R — 5 cm and h = 1 mm. If 
the torque required to rotate the disk at 900 r/min is 0.537 Nm, what is the viscosity of the fluid? If the uncertainty in 
each parameter (M, R, h, fl) is ±1 percent, what is the overall uncertainty in the viscosity? 


/ Clearance h 







t J, , 


R 1 R 


Y.4 The party is over and it is raining hard. Your car is parked several streets away. The way to your car is open, 
exposed to the rain. You are wearing your new, designer clothes. You just got the first monthly statement and it hurts. 
You want to make sure you soak them as little as possible. You have no umbrella. You are getting ready to run as 
hard as you can when all of a sudden, you start doubting whether this is the best way to save your clothes. Should 
you walk instead? The decision is too important to leave to chance. Besides, you are an engineer. You walk back into 
the building, pull out a pencil and piece of paper and start looking for the right answer. 

Y. 5 Consider an aircraft in flight. Which aerodynamic surface is working harder to generate lift: the wing or the 
horizontal stabilizer? Why? 
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